One of the key aspects of a biological memory is its ability to store and recall representations as structured sequences. Human beings and animals learn about their environment and temporally related occurrences by association within their memories. Mental items (ideas, perceptions, sensations or feelings) are connected in memory when they occur either simultaneously or in close succession, or alternatively, on the basis of similarity or contrast. Another important factor is the background or context in which perceptions occur; this, to a great extent, being responsible for the high capacity in selectivity of a human memory.
In the past several years, electronics researchers have attempted to duplicate the ability of the human brain to encode a sequence of items, and subsequently evoke that sequence by searching on the basis of similarity to an input pattern. Proposed models for mechanisms by which the temporal recall of memorized sequences can be achieved have centered around a state machine approach. Finite state machines for performing sequential operations are well-known in the field of digital computers but only recently have they been introduced into the field of analog neural networks and associative memories. For a general discussion of associative recall in a neural network, see "Self-Organization and Associative Memory", by Teuvo Kohonen, Springer-Verlag, N.Y., p. 1, 1987.
FIG. 1.10 of the Kohonen reference shows an associative memory for structured sequences. The memory, comprising a neural network having an array of synapses and neuron amplifiers, has external inputs for receiving a temporal pattern, a background or context, and a fedback pattern (which is a stored copy of a prior state of the outputs). Normally, each of these inputs would be coupled to the word line inputs of the synapse array. As explained by Kohonen, temporal recollection can be triggered if the network is excited by a particular temporal pattern in combination with a given context. An output sequence is produced, which in turn is delayed and fedback into the memory. The fedback pattern, in association with the other inputs, evokes the next pattern in the sequence, and so on.
Other prior art known to Applicant are the undated articles entitled "Attractor Dynamics and Parallelism in a Connectionist Sequential Machine" by Michael I. Jordan and "Recurrent Nets for the Storage of Cyclic Sequences" by Lvzian Wolf. Both articles generally discuss sequential behavior in a neural network but neither is considered pertinent to the present invention.
While there have been numerous proposed semiconductor cells for implementing synapses and neural amplifiers, Applicant is unaware of any proposed devices facilitating the temporal recall of structured sequences from an associative memory. Thus, what is needed is a device for providing a delayed feedback to a neural network.